1. Field
Embodiments of the invention relate to the field of digital cameras; and more specifically, to processing captured images to increase apparent sharpness.
2. Background
A camera captures an image by recording the light reflected from a subject. A camera may use an electronic image sensor, such as a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) sensor, to record images. The electronic image sensor converts light that falls on area of the sensor into an electrical charge that is proportional to the amount of light received. The electronic image sensor may include a large number of separated areas arranged in a pattern over the image sensor. The areas represent pixels of an image.
A lens focuses an optical image on the electronic image sensor. The electronic image sensor converts the light from the optical image into a pattern of charges on the image sensor. These charges may be read in the form of electrical signals that can be converted into digital representations of the light intensity for each pixel of the image sensor. The lens will not produce an image that is as sharp as the objects being photographed. For example, a sharp white to black edge in the object will have some area of gray between the white and black in the image created by the lens.
The Modulation Transfer Function (MTF) describes the resolution and performance of an optical system. MTF is the ratio of relative image contrast divided by relative object contrast (MTF=Relative Image Contrast/Relative Object Contrast). When an object (illuminated target or reticle) is observed with an optical system, the resulting image will be somewhat degraded due to inevitable aberrations and diffraction phenomena. A real lens will not fully conform with the design data. Manufacturing errors, assembly and alignment errors in the optics will further deteriorate the overall imaging performance of the system. As a result, in the image, bright highlights will not appear as bright as they do in the object, and dark or shadowed areas will not be as black as those observed in the original patterns, particularly at edges where there are rapid changes in brightness of the image. The basic terms and mathematical relations used for MTF are described in the ISO 9334 standard.
A digital camera captures a raw file, which is a record of the data captured by the sensor. A typical digital camera uses a “mosaic” or “color filter array” (CFA) sensor. Color filter array sensors use a two-dimensional area array to collect the photons that are recorded in the image. The array is made up of rows and columns of photosensitive detectors—typically using either CCD (charge-coupled device) or CMOS (complementary metal oxide semiconductor) technology—to form the image. In a typical setup, each element of the array contributes one pixel to the final image. The sensors count photons—they produce a charge that's directly proportional to the amount of light that strikes them. Raw files from color filter array sensors are grayscale.
The color filter array creates color images from the raw grayscale capture. Each element in the array is covered by a color filter, so that each element captures only red, green, or blue light. Many sensors apply the filters in a Bayer pattern where every other filter is green, in a checkerboard pattern, and the remaining filters are red and green on alternating rows as described in U.S. Pat. No. 3,971,065. Other filter patterns are possible. Regardless of the color filter arrangement, each element in the sensor captures only one color. The grayscale raw capture is converted into a color image by demosaicing. The “missing” color information for each pixel is interpolated from its neighbors to provide full color pixels with red, green, and blue values. The demosaicing process creates an additional reduction of the MTF of the image capture beyond the MTF of the lens optics.
A blur filter is generally placed over the sensor to reduce aliasing artifacts and moiré patterns due to image details that repeat with a frequency near or greater than the grid frequency of the sensor pixels. This will reduce the MTF of the image capture still further.
Image processing that provides a sharpening effect is applied to the raw file to increase the apparent sharpness of the captured image made available to the user of the digital camera. This is intended to mask the slight blurring effects of lens, filter, and sensor design and is termed “capture” sharpening. This generally involves increasing the contrast of edges in the image which viewers perceive as sharpness. However, excessive sharpening produces a halo effect that looks unnatural and degrades the image. Therefore it is important to apply only as much sharpening as is needed to create apparent sharpness.